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(54) Method for modification of non-image data in an image processing chain 


(57) A method for modification of metadata in an 
image processing chain is disclosed. The method com: 
prises the steps of: providing metadata corresponding 
to characteristics of a specific digital image; generating 
a metadata transformation related to at least one spe- 
cific image transformation; and modifying the metadata 
according to the specific image transformation. The 
method is implemented in an image reproduction sys- 
tem. The system has an imaging capture device for pro- 
viding digital image data. The imaging capture device 


can be for example a film scanner, flat bed scanner, or 
digital camera. A memory device stores metadata asso- 
ciated with the captured digital image data. The image 
transformations are carried out by an image processing 
chain. In addition to that a metadata processing chain is 
implemented and connected to the image processing 
chain in cases where the image transformations are 
metadata sensitive. 
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Description 

FIELD OF THE INVENTION 

[0001] This invention relates to a digital imaging system wherein noise and other non-image data associated with 
the input devices and media is propagated through a series of imaging transforms. Particularly, this invention relates to 
the methods for propagating the noise and other non-image data through an image processing chain. 

BACKGROUND OF THE INVENTION 


[0002] Many image processing operations require non-image information in order to properly adapt their parame- 
ters to the expected conditions. Non-image information refers to data that is carried along with each image and provides 
information about the capture conditions, devices, expected noise statistics, etc. Non-image data will herein be also 
referred to as metadata. An example of this type of image processing operation is a noise reduction algorithm that uses 
15 a table of noise root-mean-square {rms) values for each color record for every signal level. The rms value of a nominally 
uniform image area is a measure of the pixel-to-pixel variation, and may be computed from a set of N values, as: 


25 where the sample mean is 


N 


N-l 


Eq.(l) 


N 


Mx N 


, xj is the pixel value of the /^-location and o x ' s tne rms pixel-to-pixel variation. Thus if the rms value is computed for 
35 uniform areas of various signal levels, the set of these values can be seen to characterize the amplitude of image noise 
in actual scenes acquired using the same image source, as a function of image signal. 

[0003] The rms statistics could be used to adaptively discern texture from noise in localized regions of an image. 
Another example is image dependent sharpening where a measure of sharpness loss could be used along with rms 
statistics to adaptively change the level of sharpening in the image. During the processing of image information in a 

40 multistage imaging system, however, the metadata, including noise statistics, may be changed by every operation or 
transformation applied to the signal, if this transformation of metadata is not taken into account, then subsequent adap- 
tive operations will not operate as intended and system performance, usually in terms of image quality, will suffer. 
. [0004] One way to account for the transformation of the non-image information is to estimate it directly at every step 
in the imaging system. For instance, U.S. Patent No. 5,641 ,596 issued June 24, 1 997 to Gray et al., entitled "Adjusting 

45 Film Grain Properties in Digital Images", discloses a method for measuring image noise statistics based on the scan- 
ning of several uniform step patches that are not usually present in actual scenes. This estimation step could be used 
in imaging systems where the image processing operations are deterministic and there exists flexibility to process the 
set of uniform patches. However, it does not account for adaptive transformation of the noise statistics after the estima- 
tion step. 

so [0005] Another approach is U. S. Patent Application Serial No. 08/822,722 filed March 24, 1997, by Snyder et al., 
now allowed as of February 1 , 1 999, which teaches how to automatically create tables consisting of noise rms values 
as a function of signal level from real images. Again, the method does not teach how to transform the noise statistics 
based on image processing operations thereafter. 

[0006] Image noise propagation through several common imaging operations is addressed in the article by Peter 
55 D. Bums and Roy S. Berns, in Color Research and Application , entitled "Error Propagation Analysis in Color Measure- 
ment and Imaging". The analysis was not, however, applied to the transformations of noise statistics in image process- 
ing systems for use by adaptive algorithms. 

[0007] Similarly, it is very useful to provide data associated with the image capture devices or capture conditions, 
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that could be smartly used by image processing algorithms to deliver enhanced image quality. U.S. Patent No. 
5,461,440 issued October 24, 1995 to Toyoda et aL, entitled "Photographing Image Correction System,* discloses a 
photographic image correction system to correct an image on the basis of degradation information inherent in a camera 
body or lens. The degradation information is inferred from the information of the camera and lens that is recorded in the 
5 film. This method does not teach, however, how to adjust the degradation information after subsequent image process- 
ing. 

[0008] U.S. Patent No. 5,694,484 issued December 2, 1 997 to Cottrell et aL, entitled "System and Method for Auto- 
matically Processing image Data to Provide Images of Optimal Perceptual Quality", teaches a method for selecting 
proper image transforms to a given set of metadata. However, this method does not teach the adjustment of the meta- 
10 data specific to the image transform. 

[0009] The prior art presented acknowledges the usefulness of noise statistics and other metadata in image 
processing applications. However, no prior art addresses the propagation of such data in an imaging system after its 
collection. 

15 SUMMARY OF THE INVENTION 

[0010] The object of the invention is to provide a method for modification of noise and other metadata along an 
image processing chain to improve the utility of data in subsequent operations and thereby improve the system perform- 
ance and the image quality. It is necessary that the metadata be transformed after each image processing operation, 
20 so that it reflects the changes made to the image. The object is accomplished by: 

a) providing metadata corresponding to characteristics of a specific digital image; 

b) generating a metadata transformation related to at least one specific image transformation; and 

c) modifying the metadata according to the metadata transformation. 

25 - > 

[0011] A further object of the present invention is to provide an alternative method for modification of metadata in 
an image processing chain to improve the output of the captured image data after processing. The metadata should be 
applied only on demand. The object is accomplished by a method that comprises the steps of: 

30 a) providing metadata corresponding to characteristics of a specific digital image; 

b) generating an image target whose characteristics are described by the provided metadata; 

c) providing an image transformation; 

d) processing the image target through the image transformation applied to the digital image; and 

e) calculating the modified metadata from the processed image target. 

35 

[0012] It is an object of the invention to provide an image reproduction system for modifying metadata during sev- 
eral steps of image processing. This improves image data output when compared to the original digital image data. The 
object is accomplished by a system that comprises: 

40 a) an imaging capture device for providing digital image data; 

b) memory device for storing noise and other metadata associated with the digital image data; 

c) an image processing chain for carrying out image transformations; and 

d) a metadata processing chain being connected to the image processing chain in cases where the image trans- 
formations are metadata sensitive. 

45 

[0013] A further object of the invention is to provide an image reproduction system for modifying noise and other 
metadata during severaJ steps of image processing. The metadata should only be applied on demand. The object is 
accomplished by a system that comprises: 

so a) an imaging capture device for providing digital image data; 

b) memory device for storing noise and other metadata associated with the digital image data; 

c) a first processing chain for carrying out image transformations; 

d) a second processing chain for carrying out transformations on a target image; and 

e) a target image evaluator for determining metadata from the target image in the second processing chain and 
55 thereby providing input to a metadata sensitive image transformation in the first processing chain. 

[0014] No prior art provides the details of metadata propagation in an imaging system, with application to adaptive 
algorithms. Clearly, the metadata needs to be transformed after each image processing operation, so that it reflects the 
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changes made to the image. Although stochastic process theory provides the background for generating such transfor- 
mations, the theory has not been applied in a digital imaging system to derive needed input data for image transforma- 
tions. 

5 BRIEF DESCRIPTION OF THE DRAWINGS 

[0015] The subject matter of the invention is described with reference to the embodiment shown in the drawings. 

Fig. 1 is a schematic description of an image acquisition and rendering system; 
10 Fig. 2 is a schematic description of an image processing chain; 

Fig. 3 illustrates an image processing chain wherein one of the image transform utilizes one or more elements of 
metadata associated with the digital image; 

Fig. 4 illustrates the process by which a metadata transform is generated; 
Fig. 5 is a typical sensitometry correction look-up table; 
is Fig. 6 is a typical signal dependent noise table for a scanned film, wherein the noise level is shown for the colors 
red, green and blue; 

Fig. 7 is a modified noise table to compensate for sensitometry correction look-up table, wherein the standard devi- 
ation of the noise is shown for the colors red, green and blue; 

Fig. 8 is a schematic description of the propagation of localized maps for adaptive algorithms; and 
20 Fig. 9 illustrates a further embodiment of the present invention wherein the image processing chain makes use of 
evaluated target image transformations. 

DETAILED DESCRIPTION OF THE INVENTION 

25 [0016] It facilitates understanding to note that the present invention operates on a digital image. As such, a digital 
image typically consists of a number of image channels. A common number of image channels is three, one channel 
for each of the red, green, and blue image channels. A pixel value within the digital image consists of the values at a 
specific location for each channel of the digital image. Thus the value of a pixel in a digital image may be represented 
as a function of location. For instance, fo(x,y) refers to a triad of pixel values, for instance { 1 9 34 1 9} respectively refer- 

30 ring to the red, green, and blue values of the pixel located at position (x,y) in the digital image. 

[001 7] With reference to Fig. 1 , an image acquisition system 8 and an image rendering system 1 2 are the input and 
the output side respectively of an image processing chain 1 0. The image acquisition system 8 can be a film-scanner, a 
document or print scanner, or a digital camera. The image rendering system 12 may be a conventional display, a digital 
printer, a photographic printer or a digital copier. The image processing chain 10 comprises a series of image trans- 

35 forms 20 1( 20 2 20\, 20 N as shown in Fig. 2. The image transforms 20 1f 20 2 , ...,20j, ...,20 N operate on the digital 

image data, provided by the image acquisition system 8, for producing transformed digital image data. The first image 
transform 20 1 is applied to the input digital image data f 0 (x,y), generating a first transformed image ff(x,y). Each remain- 
ing image transform / (where 2 < / < N) operates upon the output of the previous image transform f h f(x t y) in order to 
generate a transformed image fj(x,y). The final image transform 20 N of the image processing chain 1 0 produces a final 

40 image f N (x,y). 

[0018] The image transforms 20 1t 20 2> ...,20|, ...,20 N of the image processing chain 1 0 may comprise simple oper- 
ations like one-dimensional look-up -tables (1D LUTs), multidimensional look-up -tables (for example a 3-dimensional 
LUT, as described in U.S. Patent 4,500,91 9 by William F. Schreiber), matrices that combine the existing channels of the 
digital image to generate new channels, spatial operations that involve linear convolution of one or more image chan- 

45 nels with a linear filter, or other more complicated transforms, like noise reduction and adaptive sharpening. 

[0019] Data associated with the digital image, which describe attributes of the digital image not described by the 
pixel values may be useful in the image processing chain 10. The data associated with attributes of a digital image not 
described by the pixel values of the digital image will herein be referred to as metadata. The set of elements of meta- 
data associated with the digital image f,(x t y) will herein be referred to as Mj. A non-inclusive list of possible elements of 

so metadata may include any of the following: image capture time, shutter speed, image color encoding metric, source of 
image (digital camera or film camera, for example), focal length of lens, lens aperture, Modulation Transfer Function 
(MTF) of capture lens, film type, detector type, subject location, subject distance, camera flash information, camera 
flash fire information, image resolution, focus information and image noise characteristics. 

[0020] With reference to Fig. 2, it may be beneficial for an image transform to have access to the image metadata. 
55 For example, in the case of a transform that sharpens an image, it is advantageous to tailor the level of sharpening by 
the metadata that describes the MTF of the capture lens. Any image transform that may be improved with accurate 
metadata describing the image f^x t y) input to the i m image transform will herein be referred to as a metadata sensitive 
image transform. An overall image quality improvement can be made by applying such image transforms with knowl- 
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edge of the image metadata. 

[0021] Furthermore, it has also been recognized that the application of certain transforms to a digital image will 
generate a transformed digital image for which the previously acquired metadata is no longer accurate. Stated another 
way, if image is transformed by the fi 1 image transform generating image ff(x,y), the metadata M M of the original 

5 image is not necessarily equivalent to the metadata of the transformed image ftflj. In essence, for each P image trans- 
form, there must be found the corresponding P metadata transform in order to maintain the correspondence between 
the metadata and the image. For example, consider again the case where the image transform sharpens the digital 
image. The metadata of the digital image may contain MTF information relating to a blurred image. A transform then 
applies a sharpening to the digital image, generating a transformed digital image. The metadata (with respect to MTF) 

w of the original digital image is then no longer accurate with respect to the transformed digital image. The advantage of 
the present invention is to create a metadata transform, related to the image transform that will modify the metadata of 
the digital image. When the metadata of the digital image is transformed by the metadata transform, the transformed 
metadata will accurately describe the attributes of the transformed digital image. Any subsequent P image transform 
that utilizes metadata will then utilize metadata that is accurate for the digital image f^x.y). 

is [0022] As another example, the input digital image data fo(x,y) may have associated metadata Mq which notes that 
the input digital image data fo(x,y) were captured with a narrow aperture. Suppose the first image transform 20 j simu- 
lates depth of field, by blurring the non-subject regions of the image, generating image f/(x,y). The appearance of image 
ffay) is that of an image captured with a wide aperture. Thus, the metadata M 0 must be transformed by a first metadata 
transform 30! to create metadata Rfl lf which then contains data associating image f£x,y) with a wide aperture. Any sub- 

20 sequent image transforms / (wherein / > 1) that make use of the aperture element of the metadata will then recognize 
that the input digital image data f(£x t y) have been modified with regard to aperture. 

[0023] Fig. 3 illustrates an image processing chain 10, of which one of the image transforms 20 1( 20 2 ,...,20 N (for 
example the third image transform 20 3 ) utilizes one or more elements of metadata associated with the digital image 
f2(x,y). The first and second metadata transforms 30<j and 302 are generated corresponding to the first and second 

25 image transforms 20j and 20 2 . Thus, M 1 Is generated by applying the first metadata transform 20j to metadata WIq. 
Likewise, Rfl 2 is generated by applying the second metadata transform 3O2 to M v Metadata M 2 describes attributes of 
the image f^y) output from the second image transform 20 2 which are not described by the individual pixels. Thus the 
third image transform 20 3 , which requires as an input some element of metadata associated with the input digital image, 
inputs the image fg(x,y) and the associated metadata M 2 which accurately describes the digital image input to the third 

30 image transform 2O3. 

[0024] Fig. 4 is an illustration of the present invention with regard to the Q+lf* image transform 20 M . The specific 
(i+lf 1 image transform 20^ is input to the metadata transform generator 40 w for generating a (i+lf* metadata trans- 
form corresponding to the (i+l)** 1 image transform. The metadata M| is then modified by the (M/ h metadata transform 
40 w for producing metadata M w , which accurately describes the characteristics of image fi+{x,y). In addition, the (i+lf 1 

35 image transform 20 w is applied to ffay) thereby generating fj+{x,y). 

[0025] The remainder of the present invention will be described with regard to one specific element of metadata, 
the image noise characteristics. Although the present invention is described with reference to noise characteristics as 
the metadata element under consideration, those skilled in the art will recognize that similar metadata transforms may 
be generated to modify other elements of metadata. The following description describes the operation of the metadata 

40 transform generator, which generates for each image transform an appropriate metadata transform for modification of 
the noise data. 

[0026] In the embodiment, shown in Fig. 4, the metadata Mj is data describing the noise statistics of the P* image 
fj(x,y). These noise statistics are described by a noise table that include the covariance between the noise in related 
(color) signals, in the following, the symbol, 07 indicates the rms value (standard deviation) for the random variable or 
45 process, r. The variance and covanance terms will be written as, e.g., c m o^. The symbols Ij and 2^, will herein indi- 
cate the covariance matrices as a function of signal level corresponding to the noise present in the images ffay) and 
fftlfcy), respectively. For notation convenience we will refer to the image ffay) as // in the following discussion. For 
three color record (r, g, b) images, the covariance matrices contain the following six quantities: o m Ogg f o^,. Org* 
Ogb f° r image data, // 

50 


55 
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[0027] For each such image transform 20 w , the corresponding transformation of the image noise statistics, repre- 
senting the f h metadata transform 3Q kl , is determined. 

£ Matrix 

[0028] With regard to Fig. 4, if the image transform 20^ is a linear matrix A, then 


10 


Eq. (2) 


75 


20 


25 


then the corresponding (metadata) noise covariance transformation is 

Z l+1 =AZiA T , Eq.(3) 

where the superscript T indicates the transpose. 
One-Dimensional Continuous Transformation 

[0029] With regard to Fig. 4, if the image transform 20j +J is a one-dimensional function. It is defined by a continuous 
function 07 for each image channel. If it is denoted as G and operates as 


30 and G can be expanded, for a three-color system, so Equation (4) becomes 


Eq.(4) 



S 


~g>(p) 

35 

t 


g*(q) 


u 




Eq.(5) 


40 where the sets of input and output signals are {p, q, r] and {s, t, u), respectively. The corresponding covariance matrix 
transform is described in the article by Peter D. Burns and Roy S. Bems, "Error Propagation Analysis In Color Meas- 
urement and Imaging." 


45 


= J G £ i+l J G 


Eq. (6) 


where 
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10 


Jr = 


where J G is evaluated at the average input signal level, u G . 


M o 

0 

dp 


o dgl 

0 

dq 


0 0 



dr 

level, n G . 



PC 


15 One-Dimensional Look-up Table 


20 


25 


30 


35 


[0030] With regard to Rg. 4, if the image transform 20 w is a one-dimensional LUX it can be seen as a discrete form 
of the above one-dimensional function. If so, the partial derivative elements of J G can be approximated by 


^ = LUT,[p)-LUT,[p^] 


or 


dp 


LUT,lp+n-LUT,[p-A) 


Eq. (7) 


Eq. (8) 


or other similar digital filter , where the discrete signals take on integer values, {0, 1 , 2, ...}. 

[0031] Fig. 5 is a typical film sensitometry correction LUT. It shows the relation between the input signal level and 
the output signal level which can be regarded as a tone scale transformation. The output signal level is not a linear func- 
tion of the input signal level. An example for the original signal dependent noise table for a scanned film is shown in Fig. 
6. Using the sensitometry correction LUT shown in Rg. 5 as the image transform under consideration, the metadata 
transform for noise characteristics is represented by equation 8. Application of this metadata transform to the original 
noise data yields the noise table in Fig. 7. The table in Rg. 7 Is a modified noise table to compensate for the sensitom- 
etry correction LUT, wherein the standard deviation of the noise is shown for the colors red, green and blue. This mod- 
ified noise table (Rg. 7) is appropriate to input to algorithms applied to the image after the sensitometry correction LUT. 
It can be noticed that Rg. 7 shows extremely high noise at low input signal levels. This is a direct result of the noise 
amplification resulting from the high slope at low input signal levels in Rg. 5. 


Multi-dimensional Continuous Transformation 

40 [0032] With regard to Rg. 4, if the image transform 20i + j is a three-dimensional function, it is defined by a continu- 
ous function for each image record. If it is denoted as G, and operates as 


45 


where 


Eq. (9) 


50 


g x (p>q>r) 

gi(P^r) 


55 


where the sets of input and output signals are {p, q, r) and {s, f, u), respectively. The corresponding covariance matrix 
transform is described in the article by Peter D. Burns and Roy S. Berns, in Color Research and Application, 22:280- 
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Eq.(10) 


where 


10 


15 


j G = 




dgi 

dp 

dq 

dr 


dg2 

%2 

dp 

dq 

dr 


dg2 


_dp 

dq 

dr J 


20 


25 


30 


35 


and J G is evaluated at the average input signal level, \i G . 
Multl-Dlmenslonal Look-up Table 

[0033] With regard to Fig. 4, if the image transform 20 j+f is a multi-dimensional LUT, it can be seen as a discrete 
form of the above continuous function. If so, the partial derivative elements of J G can be approximated by discrete dif- 
ferences, e.g., 

d9 -l = LUT,[p,q,ryLUT,[p -1,q,r] Eq. (11a) 


or 


3p 


3 gl LUTJp+l.qjHUTJp^tq.r) 
dp 2 


Eq. (11b) 


or other similar digital fitter. 

[0034] The noise propagation can be accomplished by estimating the six unique elements of the matrix. Thus, the 
element of metadata M w regarding the image noise statistics may be determined. 

40 Spatial Rlter 

[0035] Digital filtering either by a fixed-coefficient kernel or by a locally adaptive kernel is usually accomplished by 
a discrete convolution. For a general (n x m) filter with coefficients h kh wherein k=1,2,...n and /=1,2,...m. the variance 
transformation is 

45 


<*ss = 


50 


II* 

L*=i/=i 


*,/ 


Eq.(12) 


if the p color-record is filtered. Note that the filter operation, becomes an identity operation for color records where no 
spatial filtering is performed. The above equations can be expanded to include pixel-to-pixel and record-to-record cov- 
55 ariance if needed. 
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Localized Metadata Maps 

[0036] Some image transforms behave in such a way that their properties are adapted based on the characteristics 
in localized regions of the image. In this regard, these algorithms are referred to as adaptive algorithms. This results in 

5 varying of the metadata, dependent upon the original pixel value within any given region. As an example, an adaptive 
noise reduction algorithm could consist of a kernel sliding across the image and aggressively blurring regions where it 
senses that the region does not contain textures or edges, while conservatively removing noise in regions where there 
are textures or edges present. Similarly an adaptive sharpening algorithm may boost the high frequencies in busy or 
edge regions of the image while avoiding the boost in the uniform backgrounds to avoid amplification of the noise. 

10 [0037] The metadata corresponding to images that have been processed using adaptive algorithms will be affected 
in the localized regions, rather than on a global basis. That is, different regions of the image will exhibit different noise 
statistics after an adaptive noise reduction algorithm has been applied. Also, different regions of the image will exhibit 
different effective sharpness after an adaptive sharpening algorithm has been applied. Another example of localized 
operations is lens falloff compensation. Lens falloff in the image capture device causes the image to exhibit different 

15 sharpness and exposure characteristics from the center of the image towards the comers. The lens light falloff could be 
compensated using an image masking transform, but the noise characteristics after application of the masking trans- 
form will be location dependent. Fig. 8 presents a schematic diagram with the preferred method to deal with the propa- 
gation of localized metadata in the present embodiment Referring to Rg. 8, a first image transform 20! is applied to the 
original image f 0 . A second image transform 20 2 is then applied. This transform is adaptive or localized as defined 

20 above. The metadata Mq corresponding to the original image is first transformed by a global metadata transformation 
producing metadata M v Due to the adaptive nature of the second image transform 20 2 a two dimensional map, L 2 (x,y), 
is created by a localized metadata transform 302- This map carries the local metadata information of the image on a 
pixel by pixel basis. In this regard the maps, Lrfx,y), are generally referred to as focalized metadata maps, or localized 
noise data maps in the case where the element of metadata is image noise. The localized map is transformed after 

25 each metadata transformation 302, 30 3« — > 30 N corresponding to image transforms 2O2, 2O3, 20 N . Noise transforms 
such as the ones described before (LUTs, Matrices, spatial filters, etc.) are applied to the localized metadata maps. 
The next example provides a detailed description of this method as it relates to the application of an adaptive noise 
reduction algorithm to a digital image. 

[0038] A popular adaptive noise reduction algorithm is considered here: Lee's sigma filter (see, Lee, J. Digital 
30 Image Smoothing and the Sigma Filter. Computer Vision, Graphics, and Image Processing 24, 255-269, 1 983). It is not 
the purpose of the authors to make claims on this particular algorithm, but to use it for demonstration purposes. The 
sigma filter consists of a rectangular sampling window that is convolved with the digital image. A sliding window is 
moved across the image, and at every center pixel position the image noise is smoothed by averaging only those neigh- 
borhood pixels that have intensities within a fixed range from the center pixel intensity. Consequently, image edges are 
35 preserved and subtle details are retained. 

[0039] It is assumed that the expected noise as a function of signal level is provided in the form of a noise table as 
the one described in US patent 5,641,596, 

c = N[r-(x,y)] 

40 

where o is the noise standard deviation, and N is the noise table. 

[0040] Rg. 6 presents an example of the noise table for scanned negative film. The sigma filter will select those pix- 
els in the local window that fall inside some pre-defined number of intensity standard deviations from the center pixel 
intensity. The center pixel is then replaced by the average of the selected pixels. That is, if ftfkj) identifies the pixels in 
45 a rectangular window of size m x n, at row k, column / in the digital image f h and f^Ky) is the center pixel value at row 
x, column y, in the digital image f M we can express the local filter transform of size m x n by the following equations: 
Let 

$*./= 1 » if Ufryh&WiWftffay)**) Eq. (13) 

so =6, otherwise 

where A is an intensity range based on the expected noise standard deviation given by the noise table (e.g. A = 2o). 
Then, 

55 
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n+x m+y 

Z YfiufM 


Z Z**/ 

k=x-n /=y-m 


[0041] After applying the adaptive noise reduction algorithm to the image, the previously acquired metadata ele- 
ments corresponding to noise characteristics, provided in the noise table, are no longer valid. Therefore, a metadata 
is transform needs to be determined to modify the noise elements in the metadata. Let o pp (d) denote the noise variance 
as a function of signal level d corresponding to image f f in Equation 14; and let Oss(x,y,d) denote the noise variance as 
a function of position (x,y) and signal level d corresponding to image f M in Equation 14 after applying an nxm sigma 
filter; due to the adaptive nature of the filter the new variance will be also a function of position, and is given by 

20 o ss (x,y,d)= o(x,yJ-a ff (d) Eq. (15) 

where the location dependent coefficient a is derived from Equation 12, and is given by 

n m 

«(*,/)= I IX/ 2 E <i-( 16 > 

25 . /f=1 /=1 

where 

= ~n~m E *< 17 > 

ZZ**./ 

*=!/=! 

35 and 8 is given by equation 1 3. 

[0042] Then, the noise at every image plane could be propagated via the two dimensional function, ct(x,y), where 
each location (x,y) corresponds to a location (x,y) in the digital image t^x,y). In essence, a(x,y) along with the original 
noise variance statistics Opp(d), become the localized metadata or noise data map L(x,y). The entries at every (x,y) 
location in L(x,y) when multiplied by the original noise variance statistics a pp (d) t correspond to the new noise variance 

40 statistics as a function of signal level, in the form of a noise table. Also, the map L(x,y) could be decimated or com- 
pressed to a manageable size in order to avoid demanding memory storage and to improve system's throughput Sub- 
sequent noise sensitive transformations will read the map, interpolate or decompress it and calculate the expected 
noise at every image location (x,y). The method could be easily extended to provide any metadata information such as 
Noise Power Spectrum, covariance, MTF, etc. on a localized basis. 


Alternath/e Embodimen t Using Generated Targets 


[0043] Fig. 9 illustrates an alternative embodiment of the present invention in which a target image go(x,y) is gen- 
erated exhibiting the characteristics of the metadata Mq. In this regard, the metadata M 0 is input to the target image 
so generator 50. In the preferred embodiment, the metadata M 0 includes noise data. This noise data is density dependent 
and may be described in a noise table. Such a table consists of a list of mean intensities and corresponding covari- 
ances. Table 1 presents an example of such table with noise standard deviation as a function of mean intensity. 

55 
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Table 1 


5 


Noise Table 

Mean Intensity 

Channel Variance 

500 

20 

505 

22 

510 

25 

515 

23 


[0044] The target image generator 50 then generates a target image go(x,y) with the noise characteristics 
15 described by Mq. In the preferred embodiment, the target Image go(x,y) is an image consisting of several patches that 
span the signal range, each of which has dimensions A pixels by B lines and statistics similar to those of the actual 
image. Each patch of A x B pixels is generated by randomly assigning each of the pixel values a value x, where x is a 
random variable with probability distribution as in the equation (18): 


20 


25 


-0.5(x-p) 2 


e 


2 


f ' {x) —7W Eq(l8) 


30 


35 


40 


45 


[0045] The value of u, for a given patch n of the target image go(x,y) may be selected as the ji value for row n of the 
noise table in Table 1 . For example, for the first line of the target image go(x,y), u. = 500, and a = 20. 
[0046] Again referring to Fig. 9, there is illustrated an image processing chain 10 for the image f 0 (x,y) and a corre- 
sponding target image processing chain 52 for the target image g(x,y). In the preferred embodiment, each image trans- 
form 52 v 52 2 , 52 N of the target image processing chain 52 is identical to the image transform 20^ 20 2 ,..., 20^, in the 
image processing chain 1 0. The first and second image transform 20j and 20 2 are then applied successively to the 
image fo(x,y). The third image transform 20 3 is a metadata sensitive image transform. The required inputs for the third 
image transform 2O3 are the image i^x t y) and the metadata M2. The metadata M 2 is determined by applying the first 
two image transforms 52j and 52 2 of the target image processing chain 52 to the target image go(x,y). The target image 
g2(x,y) output from the second target image transform 52 2 is then passed to a target image evaluator 54 for generating 
metadata The target image evaluator 54 computes the mean u. and covariance elements of each line in the target 
image g^(x,y), thereby generating the required information for noise table of metadata M 2 . The metadata M 2 may then 
be passed to the third image transform 2O3 along with the image fefcyj for generating the image f^(x t y). Those skilled 
in the art will recognize that this process may be continued for any length of image processing chain. 
[0047] The invention has been described in detail with particular reference to certain preferred embodiments 
thereof, but it will be understood that variations and modifications can be effected within the spirit and scope of the 
invention. 
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image acquisition system 
image processing chain 
image rendering system 
image transforms 
metadata transform 
metadata transform generator i+l 
target image generator 
target image processing chain 
target image transforms 


11 



EP1 063 611 A2 


54 

target image evaluator 

fo(x,y) 

input digital image data 

f**.y) 

first transformed image 

M x >y) 

final image 

5 M, 

set of elements of metadata i 

Wx,y) 

input to the image transform i 

Mx,y) 

localized metadata map i 

9o(*>y) 

target image 
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1. A method for modification of metadata in an image processing chain; comprising the steps of: 

a) providing metadata corresponding to characteristics of a specific digital image; 
is b) generating a metadata transformation related to at least one specific image transformation; and 

c) modifying the metadata according to the transformation as carried out in step b. 

2. The method as claimed in claim 1 , wherein the metadata comprises one or more of the following: system Modula- 
tion Transfer Function (MTF), system spectral responsifrvities, capture device aperture, system effective aperture, 

20 depth of field, or localized metadata maps. 

3. The method as claimed in claim 1, wherein the metadata transformation comprises one or more of the following: 
univariate transformations, multidimensional transformations, matrices, one-dimensional look-up-tables, multidi- 
mensional look-up-tables, linear spatial filtering operations, adaptive filtering operations or masking operations. 

25 

4. A method for modification of noise data in an image processing chain; comprising the steps of: 

a) providing noise data corresponding to noise characteristics of a specific digital image; 

b) generating a noise transformation related to at least one specific image transformation; and 
30 c) modifying the noise data according to the noise transformation. 

5. The method as claimed in claim 4, wherein the noise data comprises one or more of the following: signal dependent 
covariance statistics, signal independent covariance statistics, noise power spectrum, or localized noise data 
maps. 

35 

6. The method as claimed in daim 4, wherein the specific image transformation comprises one or more of the follow- 
ing: univariate transformations, multidimensional transformations, matrices, one-dimensional look-up-tables, multi- 
dimensional look-up-tables, linear spatial filtering operations, adaptive filtering operations or masking operations. 

40 1. k method for modification of metadata in an image processing chain; comprising the steps of: 

a) providing metadata corresponding to characteristics of a specific digital image; 

b) generating an image target whose characteristics are described by the provided metadata; 

c) providing an image transform; d) processing the target through the image transform applied to the 
45 digital image; and 

e) calculating the modified metadata from the processed target 

8. The method as claimed in claim 7, wherein the metadata comprises of one or more of the following: system Mod- 
ulation Transfer Function (MTF), system spectral responsifrvities, capture device aperture, system effective aper- 

so ture, depth of field, or localized metadata maps. 

9. The method as claimed in claim 7, wherein the metadata transformation comprises one or more of the following: 
univariate transformations, multidimensional transformations, matrices, one-dimensional look-up-tables, multidi- 
mensional look-up-tables, linear spatial filtering operations, adaptive filtering operations or masking operations. 

55 

10. A method for modification of noise data in an image processing chain; comprising the steps of: 

a) providing noise data corresponding to noise characteristics of a specific digital image; 
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b) generating an image target whose characteristics are described by the provided noise data; 

c) providing an image transform; 

d) processing the image target through the image transform applied to the digital image; and 

e) calculating the modified noise data from the processed image target 

5 

11. The method as claimed in claim 10, wherein the noise data comprises one or more of the following: signal depend- 
ent covariance statistics, signal independent covanance statistics, noise power spectrum, or localized noise data 
maps. 

io 12. The method as claimed in claim 10, wherein the specific image transformation comprises one or more of the fol- 
lowing: univariate transformations, multidimensionai transformations, matrices, one-dimensional look-up-tables, 
multidimensional look-up-tables, linear spatial filtering operations, adaptive filtering operations or masking opera- 
tions. 

15 13. An image reproduction system comprising: 

a) an imaging capture device for providing digital image data; a memory device for storing noise and other 
metadata associated with the digital image data; 

b) an image processing chain for carrying out image transformations; and 

20 c) a metadata processing chain being connected to the image processing chain in cases where the image 

transformations are metadata sensitive. 

14. The system as described in claim 13, wherein the noise and other metadata comprises one or more of the follow- 
ing: signal dependent covariance statistics, signal independent covariance statistics, noise power spectrum, local- 

25 ized metadata maps, system Modulation Transfer Function (MTF), system spectral responsitivities, capture device 
aperture, system effective aperture or depth of field. 

15. The system as described in claim 13, wherein the image processing chain comprises one or more of the following 
image processing operations: univariate transformations, multidimensional transformations, matrices, one-dimen- 

30 sional look-up-tables, multidimensional look-up-tables, linear spatial filtering operations, adaptive filtering opera- 
tions or masking operations. 

16. The system as described in claim 13, wherein the metadata processing chain has at least one metadata transfor- 
mation related to the image transformation. 

35 

17. The system as described in claim 13, wherein the metadata processing chain has at least one noise data transfor- 
mation related to the image transformation. 

18. An image reproduction system comprising: 

40 

a) an imaging capture device for providing digital image data; memory device for storing noise and metadata 
associated with the digital image data; 

b) a first processing chain for carrying out image transformations; a second processing chain for carrying out 
transformations on a target image; and 

45 c) a target image evaluator for determining metadata from the target image in the second processing chain and 

thereby providing input to a metadata sensitive image transformation in the first processing chain. 

19. The system as described in claim 1 8, wherein the noise and other metadata comprises one or more of the follow- 
ing: signal dependent covariance statistics, signal independent covariance statistics, noise power spectrum, local- 

50 ized metadata maps, system Modulation Transfer Function (MTF), system spectral responsitivities, capture device 
aperture, system effective aperture or depth of field. 

20. The system as described in claim 18, wherein the first and second image processing comprises one or more of the 
following image processing operations: univariate transformations, multidimensional transformations, matrices, 

ss one-dimensional look-up-tables, multidimensional look-up-tables, linear spatial filtering operations, adaptive filter- 
ing operations or masking operations. 
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